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ASSESSING  THE  VALIDITY  OF  THE  RIDE  MOTION  SIMULATOR 
FOR  A  REMOTE  VEHICLE  CONTROL  TASK 

John  W.  Ruffner3,  Kaleb  McDowellb,  Victor  J.  Paulc, 

Harry  J.  Zywiolc'  Todd  T.  Mortsfield3,  and  James  Gombashb 
aDCS  Corporation,  bUS  Army  Research  Laboratory  -  Human  Research  and  Engineering  Directorate, 

CUS  Army  Tank- Automotive  Research,  Engineering,  and  Development  Center 

The  lightweight,  fast-moving  design  proposed  for  operations  occurring  within  5-10  years  requires 
Soldiers  riding  as  passengers  in  moving  vehicles  to  perform  operations  previously  conducted  only  in 
stationary  environments.  Operating  under  motion  conditions  can  lead  to  performance  degradations 
associated  with  physical  perturbations  and  conflicting  sensory  inputs,  which  are  associated  with 
motion  sickness.  Full-motion  simulators  offer  the  flexibility  to  model  and  rapidly  test  multiple  vehicle 
profiles  and  crew  station  design  configurations  while  providing  increased  experimental  control. 

However,  a  major  concern  is  whether  or  not  a  simulator  can  evoke  the  behavioral  responses  observed 
in  real  life.  This  validation  study  compares  the  results  of  two  complementary  experiments  that 
examined  task  performance  while  operators  underwent  either  simulated  or  actual  vehicle  motion. 

Driving  performance  indicated  differences  between  the  experiments  for  several  measures,  while 
motion  sickness  questionnaire  subscales  indicated  similar  patterns  of  results  across  both  experiments. 

Overall,  support  was  found  for  both  absolute  and  relative  validity  of  using  the  simulator  to  examine 
issues  related  to  motion  sickness,  but  not  for  performance  measures.  Our  results  support  the  premise 
that  simulators  can  be  valuable  for  inducing  specific  types  of  real-life  behaviors  that  will  be  inherent  to 
designs  proposed  for  future  forces. 

teleoperation  while  “on  the  move”.  Adding  motion  cues 
through  the  use  of  a  motion-base  is  generally  believed  to 
increase  performance  in  vehicle  control  type  tasks  (McLane 
&  Wierwille,  1975;  Reed  et  al.,  1999).  However,  previous 
research  has  focused  on  motion  cues  that  represent  the 
motion  of  the  controlled  vehicle.  By  contrast,  a  major 
concern  to  future  force  development  is  the  situation  in 
which  the  operator  experiences  cues  that  do  not  match  that 
of  the  controlled  vehicle.  This  occurs  when  a  crewmember 
operates  a  remote  vehicle  while  seated  in  another  moving 
vehicle.  Motion-base  simulators  can  be  used  to  simulate 
such  a  scenario  in  more  controlled  laboratory  conditions. 
However,  due  to  the  inability  to  perfectly  re-create  vehicle 
motions,  a  validation  of  this  scenario  was  performed  which 
focuses  on  comparing  the  influences  of  simulated  physical 
vehicle  motion  on  task  performance  to  that  of  actual 
physical  vehicle  motion. 

Purpose 

The  objective  of  the  study  reported  in  this  paper  is  to 
examine  the  validity  of  the  Ride  Motion  Simulator  (RMS) 
(see  details  in  the  Method  section)  as  a  tool  to  examine 
human  performance  issues  within  the  Soldier  environment 
expected  for  the  future  forces.  The  specific  purpose  was  to 
establish  validity  for  the  situation  in  which  a  crewmember 
teleoperates  or  “drives”  a  remote  unmanned  vehicle  while 
seated  in  another  moving  vehicle.  This  situation  is  marked 
by  the  differences  between  the  operator’s  own  self-motion 
and  the  visual  motion  the  operator  observes  via  an  indirect 
vision  system. 

These  differences  are  expected  to  lead  to  performance 
decrements  in  driving  behavior  and  incidences  of  motion 
sickness.  This  study,  which  focuses  on  the  High  Mobility 
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INTRODUCTION 

The  lightweight,  fast-moving  design  proposed  for  future 
forces  requires  Soldiers  riding  in  moving  vehicle  to  perform 
operations  that  were  previously  conducted  only  in  stationary 
environments.  Operating  under  motion  conditions  can  lead 
to  performance  degradations  associated  with  physical 
perturbations,  sensory  inputs,  and  central  nervous  system 
functioning.  Furthermore,  operating  in  an  enclosed  moving 
environment  creates  a  situation  defined  by  potential  sensory 
mismatch,  which  can  contribute  to  cognitive  and  physical 
performance  degradation  and  motion  sickness  (Reason  & 
Brand,  1975). 

Motion-base  simulators  have  the  potential  to  safely, 
rapidly,  and  cost-effectively  test  Soldier  performance  using 
multiple  vehicle  motion  profiles  and  crew  station  design 
configurations,  while  providing  increased  experimental 
control  (Reed  &  Green,  1999).  A  major  concern  is  if  motion 
simulators  can  provide  reasonable  approximations  of  the  ride 
characteristics  experienced  by  crewmembers  for  the 
purposes  of  evoking  the  behavioral  responses  observed  in 
real  life,  i.e,  the  behavioral  validity  of  the  simulator 
(Blaauw,  1982). 

Validation 

Validity  can  be  considered  as  either  absolute  or  relative 
(Blaauw,  1982).  A  simulator  is  said  to  have  absolute  validity 
if  human-vehicle  system  performance  observed  within  the 
simulator  is  of  the  same  scale  as  that  observed  in  real  life.  A 
simulator  is  said  to  have  relative  validity  if  the  same  trend 
for  a  given  task  is  seen  within  the  simulator  and  in  real  life. 

Validity  is  also  task  specific  (Blaauw,  1982).  Future 
forces  will  perform  control  tasks  such  as  vehicle 
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Multi-Purpose  Wheeled  Vehicle  (HMMWV),  is  part  of  a 
program  of  research,  the  purpose  of  which  is  to  establish  the 
behavioral  validity  of  the  RMS  for  simulating  vehicle 
motion  for  both  performance  and  motion  sickness  measures. 

The  study  was  conducted  to  address  the  following 
research  questions: 

•  How  accurately  does  teleoperator  performance  in  the 
RMS  resemble  that  of  a  teleoperator  in  a  real  combat 
vehicle? 

■  How  similar  are  the  reports  of  motion  sickness  in  the 
RMS  environment  to  that  of  a  real  combat  environment? 

METHOD 

The  results  of  two  complementary  experiments  studying 
a  vehicle  control  type  task  within  a  simulated  HMMWV  and 
an  actual  HMMWV  were  compared.  The  first  experiment 
(Simulator  Experiment)  focused  on  performance  differences 
between  simulated  indirect  driving  and  simulated  vehicle 
teleoperation  under  several  conditions.  The  second 
experiment  (Vehicle  Experiment)  examined  the  effects  of 
different  physical  motion  environments  on  an  operator  of  a 
simulated  remote  vehicle.  As  detailed  below,  both 
experiments  followed  the  same  general  protocol  and  two  of 
the  four  conditions  in  each  experiment  were  identical  with 
the  exception  of  the  use  of  the  RMS  to  simulate  the 
HMMWV  motion  (Simulator  Experiment)  versus  the  use  of 
an  actual  HMMWV  in  the  field  (Vehicle  Experiment). 

Participants 

The  participants  (n  =  16  per  experiment)  for  the 
validation  study  were  male  and  female  volunteers  drawn 
from  soldiers  at  Detroit  Arsenal,  and  employees  from  the 
workforces  at  TARDEC  and  DCS  Corporation  (Simulator 
Experiment),  and  the  U.S.  Army  Research  Laboratory  - 
Human  Research  and  Engineering  Directorate  (ARL-HRED) 
and  DCS  Corporation  (Vehicle  Experiment)  The  voluntary, 
fully  informed  consent  of  the  persons  used  in  this  research 
was  obtained  as  required  by  32  CFR  219  and  AR  70-25 
(Code  of  Federal  Regulations,  1999).  The  investigators 
adhered  to  the  policies  for  the  protection  of  human  subjects 
as  prescribed  in  AR  70-25  (Department  of  the  Army,  1990). 
Employees  who  participated  did  not  receive  any 
compensation  other  than  their  regular  pay.  Overall,  8 
participants  (n  =  5  Simulator;  n  =  3  Vehicle)  had  experience 
driving  HMMWV's  (7.0  ±1.7  yrs  (s.e.)).  Attempts  were 
made  to  match  the  participants  for  age  (29  ±  2.3  Simulator; 
31  ±  2.0  Vehicle),  occupation  (n  =  4  military  per 
experiment),  and  gender  (female  participants:  n  =  4 
Simulator;  n  =  3  Vehicle). 

Experimental  Design 

In  both  experiments,  participants  completed  an  identical 
task  in  four  conditions.  The  task  consisted  of  participants 
driving  a  physics-based  HMMWV  model  over  a  digitized 
version  of  the  Ground  Vehicle  Experimentation  Course 
(GVEC)  located  at  Aberdeen  Proving  Ground  (see  Figure  1). 
The  primary  manipulation  in  both  experiments  was  the  type 


of  physical  motion.  In  both  experiments,  participants 
completed  the  driving  task  while  undergoing  conditioqg  in 
which  the  vehicle  they  rode  either  idled  (Idling)  or  moved  in 
a  manner  not  linked  with  the  remote  driving  task  (Moving). 
These  two  conditions,  completed  in  both  experiments,  were 
assessed  as  the  basis  for  this  validation.  In  the  Simulator 
Experiment,  the  remaining  two  conditions  involved  a 
manipulation  of  the  relationship  of  task  performance  to  the 
physical  motion.  In  the  Vehicle  Experiment,  the  influences 
of  physical  oscillations  on  task  performance  were  examined 
in  the  remaining  two  conditions.  The  order  in  which  the 
participants  experienced  the  driving  conditions  within  each 
experiment  was  counterbalanced  and  subjects  were 
randomly  assigned  across  conditions  and  orders. 


Figure  1.  Experimental  Setup.  The  upper  quadrants  show  a 
participant  in  the  replicated  HMMWV  control  stations.  The 
lower  left  quadrant  is  a  bird’s-eye  view  of  the  controlled 
vehicle,  which  is  following  the  yellow  path.  The  lower  right 
quadrant  illustrates  the  participant’s  view  during  the  task.  In 
both  lower  views  obstacles  can  be  seen  in  the  distance. 

Apparatus  and  Equipment 

The  station  for  the  remote  vehicle  control  task  included 

1)  an  18-inch  active  matrix  LCD  flat  panel  display  (1280  X 
1024  pixel  resolution,  170°  viewing  cone,  235  nit  brightness, 
350:1  contrast  ratio,  and  25  ms  typical  pixel  response  time), 

2)  a  steering  wheel,  and  3)  an  accelerator  and  brake  pedal 
assembly  located  at  the  participant’s  feet.  The  control  task 
used  the  SimCreator®  Software  package  (Realtime 
Technologies,  Inc.,  Royal  Oak,  MI),  which  simulated  the 
vehicle  dynamics  and  displayed  an  approximately  40° 
horizontal  x  30°  vertical  field-of-view  computer  generated 
color  image  of  the  forward  terrain.  The  simulated  camera 
was  located  along  the  centerline  of  the  HMMWV,  at  roof 
height  above  the  windshield.  As  the  visual  portions  of  the 
vehicle  control  task  were  not  manipulated  in  this  experiment, 
only  the  general  details  of  the  visual  simulator  are  specified. 

The  vehicle  teleoperation  station  was  located  in  either  a 
replication  of  a  HMMWV  right  rear  passenger  side  seat  (see 
Figure  1)  or  an  actual  HMMWV  seat.  In  the  Simulator 
Experiment,  the  station  replicated  the  size,  location,  and 


orientation  of  the  seat,  steering  wheel,  foot-pedals,  and 
visual  display  located  within  an  enclosed  cab.  The  cab  was 
mounted  on  top  of  the  RMS  (MTS  Systems)  (http:// 
www.tacom.army.mil/tardec/nac/teams/mbt/rms.htm)  loca¬ 
ted  at  the  Ground  Vehicle  Simulation  Laboratory  (GVSL)  at 
the  US  Army  Tank  and  Automotive  Research,  Development 
and  Engineering  Center  (TARDEC).  The  RMS,  which 
generated  the  simulated  motion,  is  a  6  degree-of-ffeedom 
(DOF)  motion-base  that  is  capable  of  simulating  the  ride 
dynamics  and  characteristics  of  a  wide  range  of  military 
ground  vehicles  traversing  a  vast  array  of  ground  surfaces  to 
include  secondary  roads  and  cross  country  terrain.  The 
platform  supports  a  reconfigurable  cab  that  is  large  enough 
to  allow  the  examination  of  crew  station  designs. 

In  both  experiments,  the  physical  vehicle  motion 
experienced,  either  simulated  or  actual,  was  generated  from 
the  vehicle  traversing  the  GVEC.  The  data  presented  in 
Table  1  show  the  differences  in  the  motion  profile  of  the 
Moving  Condition  between  experiments.  The  crest  factor 
(C.F.),  the  ratio  of  the  peak  amplitude  to  the  root  mean 
squared  error  value,  describes  the  amount  of  distortion  in  a 
fluctuating  signal.  The  differences  observed  in  the  motion 
profiles  between  the  Simulator  and  Vehicle  are  consistent 
with  expectations  due  to  the  inherent  characteristics  of  RMS 
simulation  of  HMMWV  motion  dynamics. 

Table  1.  Ride  Quality.  Root  mean  square  errors  (RMSE)  and 
crest  factors  (C.F)  for  angular  (deg/s2)  and  linear  (m/s2) 
accelerations  for  each  experiment. 


Simulator  Vehicle 


RMSE 

C.F. 

RMSE 

C.F. 

Pitch 

37.2 

11.4 

5.7 

13.2 

Roll 

33.5 

10.5 

4.8 

9.0 

Yaw 

17.8 

8.0 

n/a 

n/a 

Vertical 

1.4 

3.5 

1.3 

5.6 

Translation 

0.7 

14.1 

1.2 

1.8 

Longitudinal 

0.7 

7.2 

1.1 

7.4 

Demographic,  medical  and  experience  data  were  used  as 
a  screening  tool  in  addition  to  providing  descriptive  data.  An 
auditory  version  of  the  Motion  Sickness  Assessment 
Questionnaire  (MSAQ)  (Gianaros,  Muth,  Mordkoff,  Levine, 
&  Stern,  2001)  was  used  for  the  subjective  estimation  of 
motion  sickness.  A  physical  discomfort  questionnaire, 
adapted  from  the  Louden  Bum  Chart  (see  Wilson  &  Corlett, 
1990),  was  used  to  monitor  participant  state  for  safety 
reasons  but  was  not  analyzed.  Postural  sway  and  cognitive 
tests  were  also  administered;  however,  they  fall  outside  the 
scope  of  this  paper  and  are  not  discussed  further. 

Driving  Task 

The  driving  simulation  course  was  a  digitized  version  of 
the  GVEC  test  area  in  daylight  conditions  (see  Figure  1). 
The  course,  which  consisted  of  dirt-  and  gravel-colored  road 
segments  and  grass  covered  cross-country  terrain,  followed  a 
serpentine  route  covering  approximately  1.5  miles.  A  six 
inch  wide  yellow  stripe  marked  the  centerline  of  the  entire 


course.  During  the  test  conditions,  five  FIMMWVs  were 
placed  on  the  course  as  obstacles  that  must  be  driven  aroujid 
to  increase  the  task  difficulty  and  novelty.  The  locations  of 
the  obstacles  were  randomized  across  conditions.  Striking  an 
obstacle  did  not  influence  the  motion  of  the  platform; 
however,  the  participant  was  informed  when  a  collision 
occurred.  Participants  were  instructed,  “Drive  as  quickly  as 
possible  while  keeping  the  center  of  the  vehicle  on  the 
yellow  line.  Do  not  in  any  circumstance  hit  another  vehicle.” 

Procedure 

A  similar  procedure  was  used  to  conduct  both 
experiments.  Prior  to  the  start  of  the  experiment,  participants 
were  briefed  and  provided  informed  consent.  They 
completed  training  and  baseline  measurements  on  the 
.  questionnaires  and  were  familiarized  with  the  simulators. 
The  participants  then  completed  the  test  conditions  each  of 
which  consisted  of  four  1 0-minute  driving  trials.  After  each 
trial,  the  participants  completed  the  MSAQ.  After  the  last 
trial,  the  participants  exited  the  testbed,  completed  the  exit 
interview,  were  debriefed,  and  released  when  they  were  free 
of  motion  sickness  symptoms. 

Data  Reduction 

For  the  driving  task,  dependent  measures  based  on 
speed  and  accuracy  of  lateral  position  were  derived  from  the 
database  coordinates.  The  course  was  first  segmented  into 
sections  of  like-radius.  For  each  segment  that  was  obstacle- 
free,  average  speed  (Speed)  and  the  average  lateral  deviation 
(Error)  were  computed.  Error  was  computed  as  the  root 
mean  square  of  the  position  of  the  vehicle  subtracted  from 
the  coordinates  of  the  centerline  of  the  test  track  on  a 
nearest-neighbor,  point-by-point  basis.  For  both  Speed  and 
Error,  exponential  curves  were  fit  relating  the  measure  to  the 
curve  radius  (Emmerson,  1970).  This  method  resulted  in 
coefficients  representing  the  maximum  speed  (SpeedMAX)  or 
minimum  lateral  deviation  (ErrorM[N)  and  how  the  speed  or 
the  error  changed  as  a  function  of  the  path  radius  (SpeedCc, 
Errorcc). 

The  percentage  of  errors  (%  Error)  was  computed  as  the 
number  of  times  a  participant  either  contacted  an  obstacle  or 
got  lost  while  avoiding  an  obstacle  divided  by  the  total 
number  of  obstacles  passed. 

The  four  subscales  of  the  MSAQ  were  examined  in 
accord  with  recommendations  by  Gianaros  et  al.  (2001).  The 
four  subscales  include  Gastrointestinal  (sick  to  stomach, 
queasy,  nauseated,  may  vomit),  Central  (faint-like, 
lightheaded,  disoriented,  dizzy,  spinning),  Peripheral 
(sweaty,  clammy/cold  sweat,  hot/warm),  and  Sopite-Related 
(annoyed/irritated,  drowsy,  tired/  fatigued,  uneasy). 

Numerous  participants  dropped-out  of  the  study.  The 
differences  between  participants  who  completed  the  test  and 
those  who  dropped-out  were  statistically  analyzed. 

Statistics 

The  statistical  models  were  represented  by  mixed  linear 
models  in  SPSS®  12.0  (Norusis,  2004).  For  all  measures,  the 
models  included  Device  (Simulator,  Vehicle),  Motion 


(Moving,  Idling),  and  Group  (Drop-Out,  Complete)  and  the 
interactions  between  these  factors  as  fixed  effects. 
Participants  within  group  was  treated  as  a  random  effect. 
The  covariance  structure  was  variance  components.  Means 
and  standard  errors  are  reported. 

RESULTS 

Both  experiments  were  marked  by  similarly  high  drop¬ 
out  rates  (Simulator  Experiment:  12  of  16  participants; 
Vehicle  Experiment:  13  of  16).  Moreover,  Figure  2 
illustrates  that  for  both  of  the  experiments,  the  majority  of 
the  drop-outs  occurred  in  the  conditions  excluded  from  the 
present  study.  Overall,  the  average  drop-out  time  was  45.1  ± 

10.8  and  26.9  ±  5.2  min  in  the  Simulator  and  Vehicle 
Experiments  respectively.  Within  only  the  conditions 
included  in  this  study,  the  drop-outs  averaged  34.1  ±  3.2  and 

30.8  ±  3.0  min  in  the  Simulator  and  Vehicle  Experiments 
respectively.  All  but  one  drop-out  in  each  condition  reported 
that  they  dropped  out  due  to  feeling  ill.  Because  of  the  drop¬ 
outs,  fewer  than  the  maximum  number  of  participants 
performed  the  Idling  (n  =  8,  Simulator,  n  =  6  Vehicle)  and 
the  Moving  (n  =  8  Simulator,  n  =  6  Simulator)  conditions. 


Idling  Moving  Other 

Condition 


Figure  2.  The  percentage  of  drop-outs  per  condition.  Drop¬ 
out  percentages  were  calculated  as  ratios  of  the  Drop-outs  to 
the  total  number  of  participants.  “Other”  indicates  the  two 
conditions  in  each  experiment  not  included  in  this  analysis. 

Driving  Performance 

Significant  Device  main  effects  (p’s  <  .05)  were  found 
for  SpeedMAX,  Speedcc,  ErrorM[N,  and  %  Error.  The  speed 
and  error  results  are  consistent  with  a  speed-accuracy  trade 
off  between  experiments.  Specifically,  in  the  Vehicle 
Experiment  participants  drove  faster  but  less  accurately,  than 
in  the  Simulator  Experiment.  Interactions  (p’s  <  .05)  were 
observed  for  ErrorMiN  including  Motion  by  Group  and 
Device  x  Motion.  The  Motion  x  Group  interaction  indicates 
a  similar  pattern  between  the  Drop-Out  and  Complete 
Groups  for  the  Motion  manipulation  across  Device  (Figure 
3).  However,  the  Moving  condition  was  associated  with 


greater  errors  in  the  Vehicle  Experiment  than  in  the 
Simulator  Experiment  (Figure  3).  t 
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Figure  3.  Motion  x  Group  and  Device  x  Motion  interactions 
for  ErrorMiN-  The  third-order  interaction  was  not  significant. 

Motion  Sickness  Ratings 

Significant  Motion  main  effects  (p’s  <  .05)  were 
observed  for  the  MSAQ  Gastrointestinal,  Peripheral,  and 
Sopite-Related  subscales.  Figure  4,  which  shows  data  for  the 
Gastrointestinal  Subscale,  illustrates  the  general  finding 
across  all  three  of  these  subscales  of  greater  effects  for 
Moving  than  for  Idling  in  both  Experiments.  For  the 
Peripheral  subscale,  a  Motion  x  Group  interaction  (p  <  .05) 
was  observed  indicating  the  Moving  condition  only 
influenced  the  Drop-out  group,  and  that  occurred  in  both 
experiments  (Figure  5). 
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Figure  4.  Motion  x  Device  interaction  for  the 
Gastrointestinal  subscale. 

DISCUSSION 

In  this  study,  we  assessed  the  validity  of  using  a  motion- 
base  to  simulate  the  physical  motion  of  a  HMMWV  during 
an  evaluation  of  human  performance  while  on  the  move. 


Overall,  support  was  found  for  both  the  absolute  and  relative 
validity  of  using  the  simulator  to  examine  issues  related  to 
motion  sickness.  Clear  differences  across  conditions  were 
observed  in  several  motion  sickness  subscales.  Most 
important,  similar  differences  were  observed  for  both  the 
scale  and  the  pattern  of  the  results  in  the  simulated  and  the 
actual  motion  environments. 
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Figure  5.  Motion  x  Device  x  Group  interaction  for  the 
Peripheral  Subscale. 

The  performance  data  did  not  lend  clear  support  for 
RMS  simulator  validity.  The  higher  speed  and  lower 
accuracies  in  the  Vehicle  Experiment  suggest  that 
participants  drove  with  a  different  style  in  the  two 
experiments.  This  may  have  resulted  from  differences  in  the 
additional  two  conditions  in  each  experiment.  That  is,  in 
those  conditions  in  the  Simulator  Experiment,  participants 
experienced  feedback  from  the  task  into  the  physical  motion. 
However,  this  did  not  occur  in  the  Vehicle  Experiment. 
Furthermore,  support  for  simulator  validity  would  have  been 
difficult  to  find  considering  virtually  no  differences  were 
observed  in  the  driving  metrics  between  the  Moving  and 
Idling  conditions.  Future  efforts  should  focus  on  refining 
driving  performance  measures  and  address  the  validity  of  the 
RMS  human  performance  measures  in  real  situations. 

CONCLUSIONS 

Motion-base  simulators  can  be  used  to  examine  future 
forces  tasks  such  as  command  and  control  that  must  be 
performed  on  the  move.  Our  results  here  suggest  that 
simulators  such  as  the  RMS  may  be  good  for  inducing 
specific  types  of  behaviors  observed  in  real-life  behaviors. 
However,  future  work  is  needed  using  the  validation 
approach  discussed  here  to  understand  the  extent  and  the 
limits  to  which  motion-base  studies  are  useful. 
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